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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 

Listing of Claims: 

1 . (Currently Amended) A hyperlinked broadcast system comprising: 

a video source providing video information for a video program including a plurality of 
consecutive video frames; 

an annotation system tracking movement of two or more video objects appearing in each 
of the plurality of consecutive video frames from a first location to a second location and 
generating annotation data and annotation data timing information, the annotation data including 
only one mask for each video frame of the plurality of consecutive video frames, each mask 
being stamped with the corresponding annotation data timing information based on a frame time 
of the corresponding video frame, each mask including location and graphics data of two or more 
graphics images to be overlaid on the two or more video objects appearing in the corresponding 
video frame, the annotation data further including a plurality of object data packets, each mask 
further including an identifier to an object mapping table included in one or more of the plurality 
of object data packets, the object mapping table including at least one entry with an indicia 
[[from]] included in the corresponding mask identifying a particular video object, the entry 
further storing an identifier to a corresponding one of a plurality of information data structures 
included in one or more of the plurality of object data packets, the corresponding information 
data structure including information associated with the particular video object; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information, and transmitting the augmented video transmission signal to a receiver, 
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wherein said augmented video information transmission generator associates said video 
information with said annotation data using said annotation data timing information, 
wherein the receiver receiving the augmented video transmission signal is programmed, 
for each of the plurality of consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 

identify the corresponding mask if the comparison results in a match ; 
retrieve the location and graphics data of the two or more graphics images from 
the identified mas k if th e comparison r e sults in a match ; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by-frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

2. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises a vertical blanking interval insertion device. 

3. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises at least one of a vertical ancillary data insertion device and a 
digital video data multiplexer. 

4. (Original) The system of claim 1 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 
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5. (Original) The system of claim 1 wherein said video information comprises 
digital video data. 

6. (Original) The system of claim 1 wherein said video information comprises 
an analog video signal. 

7. (Original) The system of claim 1 further comprising: 
a post production environment; and 

a headend comprising said augmented video information transmission generator, 
wherein said video information and said annotation data timing information are combined 
by said post production environment and transmitted to said headend. 

8. (Original) The system of claim 7 wherein said headed is a cable headend. 

9. (Original) The system of claim 7 wherein said headend is a satellite headend. 
10-13. (Canceled) 

14. (Previously Presented) The system of claim 1 wherein the receiver-displays 
said annotation data in response to a viewer request. 

15-16. (Canceled) 

17. (Previously Presented) The system of claim 1 wherein the mask comprises 

location information of the two or more objects in the corresponding video frame. 
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18. (Previously Presented) The system of claim 17 wherein said location 
information includes a graphics location reference that represents a fixed relation to a set of 
pixels associated with each said object. 

19. (Original) The system of claim 18 wherein said graphics location reference 
includes an upper left most pixel in said associated pixel set. 

20. (Original) The system of claim 18 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

21. (Previously Presented) The system of claim 1 wherein the mask comprises 
location and shape information of the two or more objects in the corresponding video frame. 

22-34. (Canceled) 

35. (Currently Amended) A method for generating a hyperlinked video signal 
comprising: 

receiving video information for a video program including a plurality of consecutive 
video frames; 

tracking movement of two or more video objects appearing in each of the plurality of 
consecutive video frames from a first location to a second location and generating annotation 
data and annotation data timing information, the annotation data including only one mask for 
each video frame of the plurality of consecutive video frames, each mask being stamped with the 
corresponding annotation data timing information based on a frame time of the corresponding 
video frame, each mask including location and graphics data of two or more graphics images to 
be overlaid on the two or more video objects appearing in the corresponding video frame, the 
annotation data further including a plurality of object data packets, each mask further including 
an identifier to an object mapping table included in one or more of the plurality of object data 
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packets, the object mapping table including at least one entry with an indicia [[from]] included in 
the corresponding mask identifying a particular video object, the entry further storing an 
identifier to a corresponding one of a plurality of information data structures included in one or 
more of the plurality of object data packets, the corresponding information data structure 
including information associated with the particular video object; 

communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator; 

synchronizing said video information with said annotation data in response to said 
annotation data timing information by said augmented video information transmission generator 
and generating an augmented video transmission signal comprising said annotation data, said 
annotation data timing information, and said video information; 

transmitting the augmented video transmission signal to a receiver, wherein the receiver 
receiving the augmented video transmission signal is programmed, for each of the plurality of 
consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 

identify the corresponding mask if the comparison results in a match; 
retrieve the location and graphics data of the two or more graphics images from 
the identified mask if th e comparison r e sults in a match ; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by-frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

36. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises a vertical blanking interval insertion device. 
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37. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises at least are of a vertical ancillary data insertion 
device and a digital video data multiplexer. 

38. (Original) The method of claim 35 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

39. (Original) The method of claim 35 wherein said video information comprises 
digital video data. 

40. (Original) The method of claim 35 wherein said video information comprises 
an analog video signal. 

41. (Original) The method of claim 35 further comprising inserting said 
annotation data timing information in a vertical blanking interval of an analog video signal. 

42. (Original) The method claim 35 further comprising inserting said annotation 
data timing information in a vertical ancillary data region of a digital video signal. 

43 . (Original) The method of claim 3 5 wherein said communicating step 
comprises transmitting said timing information and said video information to a broadcast 
network and subsequently to said augmented video information transmission generator. 



44-45. (Canceled) 
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46. (Previously Presented) The method of claim 35 wherein the mask 
comprises location information of the two or more objects in the corresponding video frame. 

47. (Previously Presented) The method of claim 46 wherein said location 
information includes a graphics location reference that represents a fixed relation to a set of 
pixels associated with each said object. 

48. (Original) The method of claim 47 wherein said graphics location reference 
includes an upper left most pixel of said associated pixel set. 

49. (Original) The method of claim 48 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

50. (Previously Presented) The method of claim 35 wherein the mask 
comprises location and shape information of the two or more objects in the corresponding video 
frame. 

51. (Previously Presented) The method of claim 50 wherein said shape 
information is represented by a graphical overlay of each said object. 

52. (Previously Presented) The method of claim 50 wherein said shape 
information is represented by an outline of each said object. 

53-55. (Canceled) 

56. (Previously Presented) The system of claim 1 further including a television 
receiver, the television receiver being configured to: 



Appln No. 09/943,583 

Amdt date March 11, 2008 

Reply to Office action of December 1 1 5 2007 

overlay a graphics image on a particular video frame for the particular video object based 
on the graphics data included in the corresponding mask; 

retrieve the identifier of the object mapping table from the corresponding mask 
responsive to a user selection associated with the overlaid graphics image; 

retrieve the object mapping table based on the retrieved identifier; 

identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid; 

locate the entry in the object mapping table with the identified indicia; 

retrieve from the located entry the identifier of the corresponding information data 
structure; 

retrieve the information data structure identified by the retrieved identifier; and 

display information based on the retrieved information data structure on the display 

device. 

57-60. (Canceled) 

61. (Previously Presented) The system of claim 1, wherein the overlaying of 
the two or more graphics images is for alerting a viewer of interactive data associated with the 
two or more video objects prior to the viewer transmitting an interactive command with respect 
to one of the two or more video objects. 

62. (Previously Presented) The method of claim 35, wherein the overlaying of 
the two or more graphics images is for alerting a viewer of interactive data associated with the 
two or more video objects prior to the viewer transmitting an interactive command with respect 
to one of the two or more video objects. 

63. (Previously Presented) A hyperlinked broadcast system comprising: 
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a video source providing video information for a video program including a plurality of 
video frames; 

an annotation system generating annotation data and annotation data timing information, 
the annotation data including a plurality of masks and a plurality of object data packets, each 
mask corresponding to a particular video frame of a video program and including graphics data 
associated with one or more video objects in the particular video frame, each mask being 
associated with the corresponding annotation data timing information and with an identifier to an 
object mapping table included in at least a particular one of the plurality of object data packets, 
the object mapping table including an entry associated with each of the one or more video 
objects in the particular video frame, each entry in the object mapping table storing an identifier 
to a corresponding one of a plurality of information data structures included in one or more of 
the plurality of object data packets, the corresponding information data structure including 
information associated with the corresponding video object; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information, and transmitting the augmented video transmission signal to a receiver, wherein 
said augmented video information transmission generator associates said video information with 
said annotation data using said annotation data timing information. 

64. (Previously Presented) The hyperlinked broadcast system of claim 63, 
wherein the receiver receiving the augmented video transmission signal is programmed to: 

compare the annotation data timing information for a particular mask with a frame time 
of a current video frame; 

retrieve the graphics data from the particular mask in response to a match; and 

overlay one or more graphics images generated based on the retrieved graphics data on 
the one or more video objects appearing in the current video frame. 
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65. (Previously Presented) The system of claim 64, wherein the receiver is 
further programmed to draw the one or more graphics images on a frame-by- frame basis based 
on the graphics data included in a plurality of masks, the drawing of the one or more graphics 
images being synchronized to the corresponding video frame based on the annotation data timing 
information associated with the plurality of masks. 

66. (Previously Presented) The system of claim 65, wherein the receiver is 
further configured to: 

receive a user selection associated with one of the overlaid graphics images for a 
particular video frame; 

retrieve the identifier of the object mapping table from the mask corresponding to the 
particular video frame responsive to the user selection; 

retrieve the object mapping table based on the retrieved identifier; 

locate the entry in the object mapping table for the video object associated with the user 
selection; 

retrieve from the located entry the identifier of the corresponding information data 
structure; 

retrieve the information data structure identified by the retrieved identifier; and 

display information based on the retrieved information data structure on a display device. 

67. (Previously Presented) A method for generating a hyperlinked video signal 
comprising: 

receiving video information for a video program including a plurality of video frames; 

generating annotation data and annotation data timing information, the annotation data 
including a plurality of masks and a plurality of object data packets, each mask corresponding to 
a particular video frame of a video program and including graphics data associated with one or 
more video objects in the particular video frame, each mask being associated with the 
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corresponding annotation data timing information and with an identifier to an object mapping 
table included in at least a particular one of the plurality of object data packets, the object 
mapping table including an entry associated with each of the one or more video objects in the 
particular video frame, each entry in the object mapping table storing an identifier to a 
corresponding one of a plurality of information data structures included in one or more of the 
plurality of object data packets, the corresponding information data structure including 
information associated with the corresponding video object; 

communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator; 

associating said video information with said annotation data in response to said 
annotation data timing information and generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information; and 

transmitting the augmented video transmission signal to a receiver. 

68. (Previously Presented) The method of claim 67 further comprising: 
comparing the annotation data timing information for a particular mask with a frame time 

of a current video frame; 

retrieving the graphics data from the particular mask in response to a match; and 
overlaying one or more graphics images generated based on the retrieved graphics data 

on the one or more video objects appearing in the current video frame. 

69. (Previously Presented) The method of claim 68, wherein the overlaying 
further includes: 

drawing the one or more graphics images on a frame-by-frame basis based on the 
graphics data included in a plurality of masks, the drawing of the one or more graphics images 
being synchronized to the corresponding video frame based on the annotation data timing 
information associated with the plurality of masks. 
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70. (Previously Presented) The method of claim 69 further comprising: 
receiving a user selection associated with one of the overlaid graphics images for a 

particular video frame; 

retrieving the identifier of the object mapping table from the mask corresponding to the 
particular video frame responsive to the user selection; 

retrieving the object mapping table based on the retrieved identifier; 

locating the entry in the object mapping table for the video object associated with the user 
selection; 

retrieving from the located entry the identifier of the corresponding information data 
structure; 

retrieving the information data structure identified by the retrieved identifier; and 
displaying information based on the retrieved information data structure on a display 

device. 

71. (Previously Presented) The method of claim 1, wherein the corresponding 
information data structure is an object properties table storing one or more second entries, 
wherein each of the one or more second entries includes an information category and an 
identifier to a second one of the plurality of information data structures providing details for the 
information category. 

72. (Previously Presented) The method of claim 1 , wherein each mask includes 
a visibility bit indicative of whether the video objects appearing in the corresponding video 
frame are enabled for being visually identified for a particular video shot. 
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